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ON SOME MEAN SQUARE ESTIMATES IN 
THE RANKIN-SELBERG PROBLEM 

Aleksandar Ivic 

An overview of the classical Rankin-Sclbcrg problem involving the asymptotic formula 
for sums of coefficients of holomorphic cusp forms is given. We also study the function 
A{x;^) (0 < ^ < 1), the error term in the Rankin-Selberg problem weighted by ^-th 
power of the logarithm. Mean square estimates for A(a:;^) are proved. 



1. THE RANKIN-SELBERG PROBLEM 

The classical Rankin-Selberg problem consists of the estimation of the error 
term function 

(1.1) A{x) := ^c„-Ca;, 

where the notation is as follows. Let ip{z) be a holomorphic cusp form of weight 
K with respect to the full modular group SL(2,Z), and denote by a{n) the n-th 
Fourier coefficient of (p{z) (see e.g., R.A. Rankin [15] for a comprehensive account). 
We suppose that ip{z) is a normalized eigenfunction for the Hecke operators T(n), 
that is, a(l) = 1 and T{n)ip — a{n)(p for every n £ N. In (1.1) C > is a suitable 
constant (see e.g., [9] for its explicit expression), and c„ is the convolution function 
defined by 

/ n \ " 
"■\~ 



The classical Rankin-Selberg bound of 1939 is 
(1.2) A(a;) = 0(a;3/5), 

hitherto unimproved. In their works, done independently, R.A. Rankin [16] derives 
(1.2) from a general result of E. Landau [11], while A. Selberg [17] states the result 

2000 Mathematics Subject Classification: 11 N 37, 11 M 06, 44 A 15, 26 A 12 
Keywords and Phrases; Tlie Rankin-Selberg problem, logarithmic means, Voronoi type formula, func- 
tional equation, Selberg class, mean square estimates 



1 



2 



Aleksandar Ivic 



with no proof. Although the exponent 3/5 in (1.2) represents one of the longest 
standing records in analytic number theory, recently there have been some develop- 
ments in some other aspects of the Rankin-Selberg problem. In this paper we shall 
present an overview of some of these new results. In addition, we shall consider the 
weighted sum (the so-called Riesz logarithmic means of order ^), namely 

(1-3) fTT^T) E ^» log^ © ■■= + ^(^; > 0), 

n<,T 

where C is as in (1.1), so that A(a;) = A(a;;0). The effect of introducing weights 
such as the logarithmic weight in (1.3) is that the ensuing error term (in our case 
this is A(a;;^)) can be estimated better than the original error term (i.e., in our 
case A(a;;0)). This was shown by Matsumoto, Tanigawa and the author in [9], 
where it was proved that 

(1.4) A(a;;0 <e a;(3-2€)/5+^ (0 < ^ < 3/2). 

Here and later e denotes arbitrarily small constants, not necessarily the same ones 
at each occurrence, while a <Ce h means that the constant implied by the <C-symbol 
depends on e. When ^ = we recover (1.2) from (1.4), only with the extra 'e' factor 
present. In this work we shall pursue the investigations concerning A(a;;^), and 
deal with mean square bounds for this function. 



2. THE FUNCTIONAL EQUATIONS 

In view of (1.1) and (1.2) it follows that the generating Dirichlet series 

oo 

(2.1) Z{s) := ^c„n-^ {s = a + it) 

n=l 

converges absolutely for cr > 1. The arithmetic function c„ is multiplicative and 
satisfies c„ n^. Moreover, it is well known (see e.g., R.A. Rankin [14], [15]) that 
Z{s) satisfies for all s the functional equation 

(2.2) r(s + K- l)T{s)Z{s) = {2n)^'-^r(K - s)T{l - s)Z{l - s), 

which provides then the analytic continuation of Z{s). In modern terminology Z{s) 
belongs to the Selberg class S of L-functions of degree four (see A. Selberg [18] and 
the survey paper of Kaczorowski-Perelli [10]). An important feature, proved by G. 
Shimura [19] (see also A. Sankaranarayanan [16]) is 



(2.3) 



OO 
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where B{s) is holomorphic for a > 0, 6„ <Ce (in fact J2n<x — a^log'^x holds, 
too). It also satisfies the functional equation 

B(s)Ai(s) = S(l-s)Ai(l-s), 

Ai(s) = 7r-3«/2r(i(s + «;-l))r(i(s + K))r(i(s + Ac + l)), 

and actually B{s) G iS with degree three. The decomposition (2.3) (the so-called 
'Shimura lift') allows one to use, at least to some extent, results from the theory of 
C(s) in connection with Z{s), and hence to derive results on A(a;). 

3. THE COMPLEX INTEGRATION APPROACH 

A natural approach to the estimation of A{x), used by the author in [8], is to apply 
the classical complex integration technique. We shall briefly present this approach 
now. On using Perron's inversion formula (see e.g., the Appendix of [3]), the residue 
theorem and the convexity bound Z{s) -C^ 1^^"^'^+^ (0 < cr < 1, \t\ > 1), it follows 
that 



If we suppose that 

(3.2) / \B{l+it)\^dt^,X^+' {e>i), 

and use the elementary fact (see [3] for the results on the moments of \({^ + it)\) 
that 

(3.3) / \C{^ + it)\^dt^X\ogX, 

Jx 



then from (2. 3), (3. 2), (3. 3) and the Cauchy-Schwarz inequality for integrals we ob- 
tain 

|Z(i+it)|dt«,X(l+«)/2+^ 

IX 



f 

Jx 



Therefore (3.1) gives 

(3.4) A(a;) «e x'{x'^/^T'^'^-^'^ + xT'^) «e x^+^ 
with T = x^/^^+'^\ This was formulated in [8] as 

THEOREM A. If 9 is given by (3.2), then 

(3.5) A(a;) <e x-^+^. 
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To obtain a value for 9, note that B{s) belongs to the Selberg class of degree 
three, hence B{^+it) in (3.2) can be written as a sum of two Dirichlet polynomials 
(e.g., by the reflection principle discussed in [3, Chapter 4]), each of length X^/^. 
Thus by the mean value theorem for Dirichlet polynomials (op. cit.) we have 
6 < 3/2 in (3.2). Hence (3.5) gives (with unimportant e) the Rankin-Selberg 
bound A(x) x"^^^^^ . Clearly improvement will come from better values of 9. 
Note that the best possible value of 9 in (3.2) is 9 — 1, which follows from general 
results on Dirichlet series (see e.g., [3, Chapter 9]). It gives 1/2 + e as the exponent 
in the Rankin-Selberg problem, which is the limit of the method (the conjectural 
exponent 3/8 + e, which is best possible, is out of reach; see the author's work [4]). 
To attain this improvement one faces essentially the same problem as in proving 
the sixth moment for \<^[^ + it)\, namely 

r \c{i+it)\''dt «, Tl+^ 

Jo 

only this problem is even more difficult, because the arithmetic properties of the 
coefficients are even less known than the properties of the divisor coefficients 

d3(n) = E 1' 

a6c=n;a,6,cGN 

generated by C^{s). If we knew the analogue of the strongest sixth moment bound 

r \C{^ + it)f dt <^ T^^Hog^ T (C>0), 
Jo 

namely the bound (3.2) with 6 = 5/4, then (3.1) would yield A(a;) x^^^+^, 
improving substantially (1.2). 

The essential difficulty in this problem may be seen indirectly by comparing 
it with the estimation of A4(a;), the error term in the asymptotic formula for the 
summatory function of ^4(71) = '^abcd=n-a b c de^ ^' "^^^ generating function in 
this case is C^(s). The problem analogous to the estimation of A(a;) is to estimate 
A4(a;), given the product representation 

00 00 
(3.6) 5^d4(n)n-« = C(s)G(s)=C(s)5^5(n)n-« (^ > 1) 

n— 1 n— 1 

with g{n) <Ce n'^ and G{s) of degree three in the Selberg class (with a pole of order 
three at s = 1). By the complex integration method one gets A4(a;) a;^/^+^ 
(here 'e' may be replaced by a log-factor) using the classical elementary bound 

/o IC(5 + it)\^dt < Tlog^'T. Curiously, this bound for A4(x) has never been 
improved; exponential sum techniques seem to give a poor result here. However, if 
one knows only (3.6), then the situation is quite analogous to the Rankin Selberg 
problem, and nothing better than the exponent 3/5 seems obtainable. The bound 
A(a;) x^l'^^^ follows also directly from (3.1) if the Lindelof hypothesis for Z(s) 
(that + ii) \t\^) is assumed. 
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4. MEAN SQUARE OF THE RANKIN-SELBERG 
ZETA-FUNCTION 



Let, for a given cr e K, 

(4.1) /x(cr) = limsup 

t^oo log t 

denote the Lindelof function (the famous, hitherto unproved, Lindelof conjecture 
for C,{s) is that ^{a) = for a > |, or equivalently that C(| + it) \t\'^)- In [8] 
the author proved the following 

THEOREM B. // /3 = 2/(5 - then for fixed a satisfying ^ < a < 1 we 

have 

(4.2) / |Z(c7 + ii)|2di = TV4n-2- + Oe(T(2-2-)/(i-/3)+-). 

This result is the sharpest one yet when a is close to 1. For a close to \ 
one cannot obtain an asymptotic formula, but only the upper bound (this is [7, eq. 
(9.27)]) 

\Z{a + it)\^ dt <e T2''(V2)(l-<-)+e(y + r3(l-a)-) ( 1 < < 1). 

The upper bound in (4.3) follows easily from (2.3) and the fact that, as already 
mentioned, B{s) G S with degree three, so that + it) can be approximated by 
Dirichlet polynomials of length <C t^^^, and the mean value theorem for Dirichlet 
polynomials yields 

J \B{a + it)fdt<^, T%T + T^^^-^y} (i<(7<l). 

Note that with the sharpest known result (see M.N. Huxley [2]) /i(l/2) < 32/205 
we obtain f3 = 410/961 = 0.426638917 ... . The limit is the value f3 = 2/5 if the 
Lindelof hypothesis (that fi{^) = 0) is true. Thus (4.2) provides a true asymptotic 
formula for 

1 + 8 1371 
a > = ——= 0.7133194.... 

2 1922 

The proof of (4.2), given in [8], is based on the general method of the author's paper 
[6], which contains a historic discussion on the formulas for the left-hand side of 
(4.2) (see also K. Matsumoto [12]). 

We are able to improve (4.2) in the case when a = 1. The result is contained 
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THEOREM 1. We have 



\Z{1 + it)\' dt = Tj2 cln-' + Oe((logT)2+^). 

n=l 

Proof. For a = ^es > 1 and X > 2 we have 

(4.5) 

n<X •'^ n<x 

/^"V-l — s /■oo 
= V CnU-' + ^ix)X-' - S / MX)X-'-^ 



By using (1.2) it is seen that the last integral converges absolutely for a = 
3?es > 3/5, so that (4.5) provides the analytic continuation of Z{s) to this region. 
Taking s = l + it,l <t <T,X = T^°,\t follows that 

(4-6) 

r \Z{1 + it)\^ At= TWY, c„n-i-* ' - 2C/9m ( ^ £!1 (^) | di + 0(1). 
By the mean value theorem for Dirichlet polynomials we have 



cln ^ 



/ |^c„n-i-fdi = T^4 + 0(^ 

^1 n<X ri<X n<X 

oo 

= T^4 + 0,((logT)^+^), 

where we used the bound (see K. Matsumoto [12]) 
(4.7) «e x{\ogxf+' 

n<x 

and partial summation. Finally we have 

("■«) Elf K^)""'*'^'"'""''^- 

n<X "'I ^ ^ 

To see that (4.8) holds, note first that for X — X/ logT <n<X the integral over 
t is trivially estimated as <C log T, and the total contribution of such n is 



«;logT y — da; < 1 

X-X/logT<n<X 
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on using (1.1)-(1.2). For the remaining n we note that the integral over t equals 



itlog{X/n) 



ilog{X/n) J I yn ) t"^ 



The contribution of those n is, using (1.1)-(1.2) again and making the change of 
variable X/u = v, 

« y -r4WT=/ , . d{Cu + A{u)) 

^ , nlog(X/n) ii_o uXogiX u) 

rX-X/losT ^ / ^^^^ ^^^^ X 

^ .X-X/XogT W-T ^^ + 1 

Jl u\og{X/u) 7(1-1/ log T)-i « log V 

= loglogX - loglog(l - l/logT)-! + 1 < loglogT, 
and (4.8) follows. 

One can improve the error term in (4.4) to ©(log^T), which is the limit of 
the method. I am very grateful to Prof. Alberto Perelli, who has kindly indicated 
this to me. The argument is very briefly as follows. Note that the coefficients 
c\ are essentially the tensor product of the c„'s, and the c„ are essentially the 
tensor product of the a(n)'s; "essentially" means in this case that the corresponding 
L-functions differ at most by a "fudge factor", i.e., a Dirichlet series converging 
absolutely for a > 1/2 and non- vanishing at s = 1. In terms of L-functions, 
the tensor product of the a{n) (the coefficients of the tensor square L-function) 
corresponds to the product of C(s) and the _L-function of Sym? (Shimura's lift). 
Moreover, Gelbart-Jacquet [1] have shown that Syw? is a cuspidal automorphic 
representation, so one can apply to the above product the general Rankin-Selberg 
theory to obtain "good properties" of the corresponding L-function. Since Sym^ 
is irreducible, the L-function corresponding to has a double pole at s = 1 and a 
functional equation of Riemann type. It follows that the sum in (4.7) is asymptotic 
to Da; log X for some D > Q, and the assertion follows by following the preceding 
argument. 

In concluding this section, let it be mentioned that, using (4.5), it easily 
follows that Z{1 + it) «: log |i| (i > 2). 

5. MEAN SQUARE OF A(a;;0 

In this section we shall consider mean square estimates for A(a;;^), defined by 
(1.3). Although we could consider the range ^ > 1 as well, for technical reasons we 
shall restrict ourselves to the range < ^ < 1, which is the condition that will be 
assumed henceforth to hold. Let 

(5.1) := inf { /3>0 : ^ A2(a;; ^ < }. 
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The definition of /3{ is the natural analogue of the classical constants in mean square 
estimates for the generalized Dirichlet divisor problem (see [3, Chapter 13]). Our 
first result in this direction is 

THEOREM 2. We have 

(5.2) ^ < < max (i^, ^) (0 < ^ < !)• 

Proof. First of all, note that (5.2) implies that /3| = (3 - 2^)/8 for i < ^ < 1, 

so that in this interval the precise value of /3g is determined. The main tool in our 
investigations is the explicit Voronoi type formula for A(a;;^). This is 

(5.3) A(ar;0 = Vi[x,N) Ri{x,N), 
where, for A/" 1, 

(5.4) 



Vi^ix, N) = (27r)-i-«x(3-2«)/8 ^ c„n-(5+20/8 cos(8n{xn)'^^ + 5(5 " 0^) > 

n<N 

R^ix, N) «, {xNY (1 + x(3-«/47V-(i+«)/4 + (a;7V)(i-«/* + x^'-^^^^) . 

This follows from the work of U. Vorhauer [20] (for ^ = this is also proved in [9]), 
specialized to the case when 

A = , B = (27^)^ M = L = 2, 61 = 62 = di = d2 = 1, /3i = K - i , 62 = i , 

5i = K - |, = -i, 7 = l,p = B, g = 4, A = 2, A = -1, C = {2tt)-^''^. 

In (5.3)-(5.4) we take A = x, so that R^{x,N) x^^-^^/^+^. Since ^ < ^ 
for ^ > ^, the lower bound in (5.2) follows by the method of [4]. For the upper 
bound we use c„ <Ce and note that {e{z) = exp(27ri^;)) 

/ I Cfe/fe-(5+2f)/8e(4(xfc)^/4)| dx 

''^ K<k<2K 

fix 

« A+ V Cfc,Cfc,(fcifc2)-(^+^«)/« / ^{^x^l\kT-kT))^x 



1/4 1,1/4 



«e x+x^/^+'K-('+^^y^ Yl 1*^1 
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where we used the first derivative tost (cf. [3, Lemma 2.1]). Since K <^ X and 

I-2X I-2X p2X 

A2(a;;0da;«; / \V(^{x,N)\'^ dx + / \R{x,N)\'^ dx, 
Jx Jx 



ix 

it follows that 



l-2X 

Jx 



IX 

which clearly proves the assertion. 

Our last result is a bound for which improves on (5.2) when ^ is small. 
This is 

THEOREM 3. We have 

(o<«<J(i + 2M4))). 

Proof. We start from 
(5.6) A(a:;C) = — lim / Z{s)^ds, 

whore < c = c(^) < 1 is a suitable constant (sec K. Matsumoto [13] for a detailed 
derivation of formulas analogous to (5.6)). By the Mellin inversion formula we have 
(see e.g., the Appendix of [3]) 

Z{s)s-^-^ = A(l/a;;Oa;'"^da; (3?es = c). 
Jo 

Hence by Parseval's formula for Mellin tranforms (op. cit.) we obtain, for /3j < 

CT < 1, 

(5.7) 

= / A2(a:;C)x-2^-Ma;>X-2<^-W A2(x;0dx. 
Jo Jx 

Therefore if the first integral converges for a = ao + £, then (5.7) gives 

I-2X 

/ A2(x;0 da; <^ X^^+\ 
Jx 

namely /3j < ctq. The functional equation (2.2) and Stirling's formula in the form 
|r(s)| = x/2^|tr-V2e— 1*1/2(1 + 0(|t|-i)) {\t\ >to>0) 
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imply that 

(5.8) Z{s) = X{s)Z{l-s), X{a+it) >; Itl^""'^ (s = a+it, < ct < 1, |t| > 2). 
Thus it follows on using (4.3) that 



I \Z{cT + it)\^At « T*-8<T / \Z{l-(j + it)\''dt 

IT JT 

^ 2i4— 8o-+2/n(i)(7+max(l,3(7)+e 



But we have 

4 - 8(7 + 2/x(i)c7 + max(l, Sci) = 4 - 5cr + 2/i(i)c7 < 2^ + 2 

for 

2 — 2^ 

(5.9) cr> (70 = - r-7TT' 

5-2^(2) 

provided that ctq > 1/3, which occurs if < ^ < g(l+2/i(i)). Thus the first integral 

in (5.7) converges if (5.9) holds, and Theorem 3 is proved. Note that this result is 
a generalization of Theorem 7 in [8], which says that /3o < (2 — 2^)/ (5 — 2/i(i)). 

In the case when /3j = (3 — 2^)/8 we could actually derive an asymptotic 
formula for the integral of the mean square of A(a;; much in the same way that 
this was done in [9] for the square of 



Ai(a;) := / A(u)du, 
Jo 



where it was shown that 
i-x 



(5.10) ^ I^l{x)dx = DX^^'^ + Oe{X^+') 

with explicit £> > (in [12] the error term was improved to Oe{X^i\ogX)^^^)). In 
the case of A(a;; 1) the formula (5.10) may be used directly, since 

(5.11) -Ai{x) = - f A{u)du = A{x;l)+OJx'). 

X X 7o 

To see that (5.11) holds, note that with c = 1 — £ we have 

1 pc+ioo s 

1 pc+ioo s 1 rc+ioc s 

= / Z{8) , ,, ds+-^ / Z{s)^r^—As 

1 1 /-e+ioo s 



X 

= -Ai{x) + 0,{x'), 

X 
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on applying (5.8) to the last integral above. 
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